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Abstract 

We establish a Julia-Caratheodory theorem and a boundary Schwarz- 
Wolff lemma for hyperbolically monotone mappings in the open unit ball 
of a complex Hilbert space. 

Let B be the open unit ball of a complex Hilbert space Ti. with inner product 
(•, •) and norm || • ||, and let p : B x B R + be the hyperbolic metric on B ([H], 
p. 98), i.e., 



p{x, y) = tanh 1 y/l - a(x,y), (1) 

where 

(1-||X|| 2 )(1-|M1 2 ) ire ^ 

We denote by N p the class of all those self-mappings F : B i— > B which are 
nonexpansive with respect to p (p-nonexpansive), i.e., 

p(F{x\F{y)\ <p(x,y). (3) 

Note that the class Af p properly contains the class Hol(B) of all holomorphic 
self-mappings of B (00). 

Definition 1 A family S = {F(t)}t>o of self-mappings of B is said to be a 
one-parameter continuous semigroup (flow) on B if 

F(t + s) = F{t)oF(s), M>0, (4) 

and 

lim F(t) = I, (5) 

where I is the restriction of the identity mapping of H. to B and the limit is 
taken pointwise with respect to the strong topology ofH. 
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We denote F(t)x, the value of F(t) at x € B, by F t (x), t > 0. 

Definition 2 A flow S — {F(t)}t>o on B is said to be generated if for each 
id!, there exists the strong limit 

f{x) := lim Ux-F t (x)\ (6) 



t->o+ t 

In this case the mapping f : B i— > is called the (infinitesimal) generator of 

S. 

If f generates a flow of p-nonexpansive self-mappings o/B, f/ien we will write 
The following result is established in |14) : 

♦ A semigroup S of holomorphic self-mappings of B is differentiable with 
respect to the parameter i > (hence, generated by a holomorphic mapping) if 
and only if it is locally uniformly continuous on B, i.e., the limit in Dchnition 1 
is uniform on each p-ball in B. 

Moreover, in this case (see [H] and |15|): 

♦ The generator f is holomorphic on B, and bounded and uniformly contin- 
uous on each p-ball in B. 

The set of all holomorphic semigroup generators is denoted by (?Hol(B). 

The classical Julia-Caratheodory theorem and the boundary Schwarz- Wolff 
lemma play a crucial role in geometric function theory (see, for example, 3 and 
|19|). In particular, they can be effectively used in the study of the asymptotic 
behavior of discrete and continuous dynamical systems. In this context these 
celebrated results may be stated as follows: 

♦ Let F be a holomorphic self-mapping of the open unit disk A in the 
complex plane C. If for a boundary fixed point r € dA ( lim F(rr) = r) the 

r— >1 - 

angular derivative ZF'M := Z lim F'(z) = k exists finitely, then 

Z — >T 

\F(z)-r\ 2 <k V~A 2 



1 - \F{z)Y ~ 1 - |z| 2 

• If k < 1 this inequality means that each horocycle internally tangent to the 
unit circle dA at r £ dA is F-invariant. 

• This is indeed the case when F has no fixed point in A and r G dA is its 
so-called Denjoy-Wolff point, that is, r is an attractive fixed point for all orbits 
ofF. 

We use the symbol Z. lim to denote the limit in each non-tangential approach 

Z — >T 

region (see, for example, [T2J and [TH]). 

Sometimes the above statements are grouped together under the name the 
Julia- Wolff- Caratheodory theorem. Higher dimensional analogs can be found, 
for instance, in [TS1 151 H fe] . 
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For holomorphic mappings on the open unit disk A in the complex plane 
C (i.e., for the one dimensional case when B = A), an infinitesimal version of 
the Julia- Wolff-Caratheodory theorem was given in |S] . Namely, it was shown 
there that 

♦ / S Q Hol(A) has no null point in A if and only if for some r € OA, the 
angular derivative 

Zf'(r) := Z lim f'(z) = a (7) 

Z — >T 

exists (finitely) with Re a > 0. 

Moreover, a is, in fact, real and if S = {F(t)} t >o is the semigroup generated 
by f, then 

\Ft(z)-T\ 2 , ,\z-t\ 2 

! < exp (-to) \ j-^ , z e A, t > 0. (8) 



l-\F t (z)\ 2 - ^ '1 

The point r is unique and a (globally) attractive sink point of S. 

It is worth mentioning that the original Julia-Caratheodory theorem deals 
not only with attractive boundary fixed points, but also with repelling fixed 
points (see [3lll6j). i.e., it deals with not necessarily fixed point free holomorphic 
self-mappings. In this direction, a generalization of the above theorem has 
recently been given by M. D. Contreras, S. Diaz-Madrigal and Ch. Pommerenke 
2 . They proved the following one-dimensional assertion: 

♦ Let f € Q Hol(A). For a boundary point r £ OA, the following claims are 
equivalent: 

(i) the angular limit Zf'M := Z lim — j3 exists finitely; 

(ii) the angular limit dFt d ^ '■= Z lim Ft ^}~ r = exp(— 1(3). 

Note that condition (ii) is equivalent to inequality JSJ for some a < (3. 

In this paper we will establish these assertions for a general complex Hilbert 
(not necessarily finite-dimensional) space Ti.. Moreover, we will show that re- 
placing the angular derivatives by just radial derivatives, we are able to prove 
infinitesimal versions of the Julia-Caratheodory theorem and the boundary 
Schwarz-Wolff lemma for the much wider class of generators of semigroups 
of p-nonexpansive self-mappings of the Hilbert ball B. For the case of the 
Denjoy-Wolff point, the asymptotic behavior of one-parameter semigroups of 
p-nonexpansive and holomorphic mappings was also studied in 0. 

Indeed, the content of the classical Schwarz-Pick lemma is the fact that 
each holomorphic self-mapping of the open unit disk in the complex plane is 
nonexpansive with respect to the Poincare hyperbolic metric p defined by (Q. 
Therefore, one can try to use metric fixed point theory to derive results regarding 
those mappings which are nonexpansive with respect to p. At the same time, 
we must remember that if a given mapping (or semigroup) is not holomorphic, 
then the notion of derivative makes no sense. 

It turns out, however, that although the study of the asymptotic behavior 
of a fixed point free semigroup consisting of p-nonexpansive mappings is, in 



3 



general, much more complicated, one can define the real part of the radial 
derivative of its generator at a boundary fixed point and use it to find invariant 
ellipsoids, internally tangent at this point to the unit sphere, in the spirit of 
the Julia-Caratheodory theorem and the boundary Schwarz- Wolff lemma. To 
understand this phenomenon, we first consider the following example. 

Example 1. Consider the continuous function / : A i— > C defined by 

a \ z + z z — z 

where % is a real parameter. Elementary calculations show that for all \ > \ 
the following boundary flow invariance condition holds: 

Re/(z)z>0, zedA. 

Therefore it follows from Martin's theorem (see also ^31) that / generates a 
semigroup of continuous self-mappings of A. Indeed, solving the Cauchy problem 

T + '<«<*» -°- (9) 

F (z) = z, z e A, 



we find 



F t (z) = l-e- t + e- t — + e~ xt ^^, z e A. 



It is clear that F t (l) = 1 for all t > and lim F t (z) = 1 for all z E A. 

t — >oo 

However, no more information on invariant sets of this semigroup can be 
obtained in this way. 

If x = lj then for each t > 0, the function Ft(z), as well as /, are holomorphic. 
Hence one can apply the result in |S] (see Q)-© above) to derive 

d ( F (z) ) - \ l - F ^)\ 2 < ^~z\ 2 (w) 
di(F t (z)) . lHz|2 < l _\ z? , (10) 

because the angular derivative of / at the boundary fixed point r = 1 exists 
and equals 1. 

But for x > 1 our generator /, as well as its generated semigroup, are again 
not holomorphic. 

However, fortunately, in this situation (x > 1) one can show that the semi- 
group S = {F t } consists of p-nonexpansive self-mappings of the open unit disk 
A with the Poincare hyperbolic metric p defined on it. Indeed, 

F t (z) = (l-e- t )+z^— 

is a convex combination of holomorphic and anti-holomorphic (hence, p-nonexpansive) 
mappings. So, by Theorem 6.5 on page 75 of 8 , Ft also is p-nonexpansive. 
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In addition, we see that although / is not differentiable in the complex sense 
in A, its radial derivative at the boundary null point t — 1 does exist: 



,. f(r) 
lira 



r^l" r — 1 

So, the following question arises: 

• Is this fact sufficient to ensure the same invariance condition as for 
a semigroup of p-nonexpansive self-mappings of the open unit disk A ? 

In this paper we give an affirmative answer to this question in a much more 
general situation. 

To formulate our results, we need the following notions and notations. 

For a fixed r G <9B, the boundary of B, and an arbitrary x £ IB, we define a 
non-Euclidean "distance" between x and r by the formula 

dr(x) = — — 3-. (11) 

1 - IN 

The sets 

f 11 _ l x T )\ 2 1 
E(r,8) = \xeB: dr(x)= l - \-L" <8 \, s>0, (12) 

are ellipsoids internally tangent to the unit sphere dM at r. 

As in PP , it can be shown that the support functional x* of the smooth 
convex set E(t, s) at x € dE(r, s), x ^ r, normalized by the condition 



lim (x — t, x*) = 1, 

X — >T 

can be expressed by 

x * = i — T!i2 x ~~i 7 ( 13 ) 

1 — \\x\\ z 1 — (r, x) 

Theorem. Let S be a semigroup of p-nonexpansive self-mappings of the Hilbert 
ball B, generated by f : B TL. Suppose that f is uniformly continuous on each 
p-ball in B, and r S dM is a null point of f in the sense that lim f(rr) = 0. 

The following assertions are equivalent: 



{I) lim sup Rc (felll) > „oo ; (u) 

r->i- r — 1 

(/(rr),r) 

(II) a := lim Re ■ — exists finitely; 

r- >l- r — 1 

(III) (3 := inf 2 Re(f{x), x*) > -oa; 
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(IV) there exists a real number 7 such that 

d T {F t (x)) < exp(-t-f) ■ d T (x), ieB. (15) 

Moreover, 

(a) a — (3 and the maximal 7 for which (IV) holds is exactly the same (3; 

(b) if f is holomorphic and one (hence all) of conditions (I) -(TV) holds, then 

r->l- J 1 

exists and is actually a real number. 

Combining this result with Theorem 8.3 in |17| . we arrive at the following 
analog of the boundary Schwarz- Wolff lemma. 

Corollary. Let S = {-Ft}t>o C Af p be a semigroup of p-nonexpansive self- 
mappings o/B generated by f £ GJ\f p (M). Assume that f is null point free (that 
is, S has no stationary point in M). Then there is a unique point r £ d~B such 
that 

d T (F t (x)) < exp(-to) • d T (x), ieB, 

for some a > 0, and there is a continuous curve {x(r) : < r < 1} C B ending 
at t £ dM for which 

lim f(x(r)) = 0. 

r— >1~ 

Conversely, if for some point t £ DM the radial limit lim /(rr) = and the 

r— ►l - 

(/(rr),r) 

radial limit a := lim Re ■ — existe and is positive, then for each t > 0, 

r— *l~ r — 1 

the mapping F t has no fixed point in B. 

To prove our theorem we need the following additional concepts and facts. 

A mapping / : B 1— > 7i is said to be hyperbolically monotone or p-monotone 
(Uni EDI) if for each pair x, y £ B, 

p^x + rf(x),y + rf(y)^j > p(x, y) (16) 

for all r > such that the points x + rf(x) and y + rf(y) belong to B. 

The crucial point in our approach is the following characterization of p- 
monotonicity |151 Theorem 2.1]: 

♦ A mapping f : B 1— * Ti is p-monotonc if and only if 



Re 



(f(x),x) , (y,f(y)) 



1-IMI 2 1-IMI 2 ) 



> Re 



(f(x),y) + (x,f(y)) 

l - (x,y) 



(17) 



for all points x, y £ B. 

Moreover, if f £ QJV P (M) is uniformly continuous on each p-ball in B, then 
it is p-monotone. 



6 



So, each holomorphic generator on B is p-monotone. 

Proof of Theorem. The implication (II)=> (I) is trivial. To prove other 
implications we denote by ip the following real-valued function: 

1>(t,x) :=d r {F t {x)). 

By direct calculation we get 

dip(t, x) 



dt 



= -2il>(0,x)Re(f(x),x*), (18) 



t=o 

where x* is defined by Hence, by the semigroup property, 
dip(t, x) 



dt 



= -2iP(t,x)Re(f(F t (x)),(F t (x))*) (19) 



for all t > 0. 

Step 1. (I)^(III). If (I) holds, then there exists a sequence {r n } such that 
r n / 1 and 

(f(rr),r) (/(r»r),r) 
Q'l := hmsupRe = hm Re . 

r _>x- r — 1 n— >oo r n — 1 

Setting y — r n r in ((T7|) . we get 

Rc (f(x),x) (f(r n T),r n T) > Re (f(x),r n T) + (x,/(r n r)) 
1 - ||a;|| 2 1 - r£ 1 — (x,r n r) 



or, equivalcntly, 



Re ( i — 7TT2 ~ i "^T" 

\ 1-|MI l-r„(r, 



> 



Letting now n tend to oo, we see that 

Re(f(x),x*)>^, 

i.e., (Ill) holds and (3 is not less than a±. By the way, this implies that ai is 
finite. 

Step 2. (III)o(IV). Let (IV) hold for some real number 7. Differentiating l|15fl 
at t = 0, we obtain 

2Re</(aO,a:*> > 7, 

i.e., (Ill) holds, and (3 > 7. 

Let now (III) hold. Then by ©J, 
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Integrating this inequality with respect to t, we see that (IV) holds with 7 = (3. 
Step 3. (III)=KI). Let 



Substituting x — rr, < r < 1, we see that 

2Re (/M'T^2-T^:)>A 

or 

r — 1 r + 1 

Therefore 

liminfRe iZ(!l)il) >^. 

r->i- r — 1 

Hence (I) holds and limsupRe (/( ^|' r) > /?. 

r-»l- 

Step 4. Just by comparing Step 1 and Step 3, we conclude that (III) implies 
(II), i.e., if (III) holds, then a = lim Re U-LELLl} exists and is equal to (3. 

Step 5. To end the proof, we have to prove (b). 

Suppose that / is holomorphic. We introduce a holomorphic function g e 
Hol(A, C) as follows: 

5 (A):=(/(At),t)-^(A 2 -1). (21) 



It follows from (III) that 



Rc</(At),(At)*)>^. 



On the other hand, 



(/(At), (At)*) = //(At), ^j— - ^ 



</(AT),T>.( r 



1 



|A| 2 1-A 



P„2 a - 1 



2 V 7 (1-|A| 2 )(1-A) 

„m l^-l| 2 1 /? (1-A)(1 + A) 

g[ ' -(1- |A| 2 )(1-A) 2 2 1- |A| 2 ' 

Therefore Re — ^^'^2 — ^" By the Riesz-Herglotz representation formula, 

g(x) - 1 1 + A W), 



-(1-A) 2 Jj C | =1 l-AC 
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where da(£) is a positive measure on the unit circle. Decomposing da with 
respect to Dirac's (^-function at the point C = 1, d<r(() = a5(() +da\(Q, a > 0, 
we calculate: 

Hm M^L = Hm (1 _ r ) I i±l£ ( a j(£) + da^Q) = 2a > 0. (22) 

r->l- r — 1 r- >1- J|^| =1 1 — rC 

This fact, in turn, implies that 

Um (Kr) = Um /aw + ^ (r + 1) A =2a + /3 (23) 

r^i- r — 1 r->i-\r — 1 2 / 
exists and is real. 

Since by Step 4, Re(2a + /?)=/?, we have that a = 0, and hence the limit 

lim <f^),r) = 
r->i- r - 1 

is real. This completes the proof of our theorem. 

Example 2. Let 7^ be a Hilbert space. Consider the semigroup of holomorphic 
self-mappings of the unit ball B of the space H := C x Hi defined by the formula 



F t (z 1 ,z 2 ) 



z\ e t/2 z 2 



z 1 +e t (l-z 1 ) ' zi + e*(l - zi) 



where z\ G C, z 2 £ Hi, |zi| 2 + ||z 2 || 2 < 1. This semigroup has a boundary fixed 
point t = (1,0) which is not its Denjoy- Wolff point. Consider the semigroup 
generator / € Hol(B,W): 



, dF t { Zl ,z 2 ) 

J\Z\,Z 2 ) = - 



(1 - 2 Zl )z 2 
= Ui(l - z x ), 



t=0 



dt 

Now we just calculate 

a= lim (fW>T) = . l . 

r^l~ r — 1 

Hence, 

d T (F t (z)) < e f d T (z), z e B, t > 0. 

Example 3. Define another semigroup {-Pi} t>0 = {(-Pt)i , O^t^tx) on tne um * 
ball B of the same space H by the formulae: 



_ (i + z 2 )e 2t - (i - gQVgq + z 2 ) e 2 < - (i - Zl y 

(Ft)l - (l + z 2 )e2t _ (1 _ Zl)2 



(F t ) 2 (zi,z 2 ) = z 21 



/ g(^) 1 (^i,^) 
9zi 
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where zi gC, z 2 G Hi, |zi| 2 + ||z 2 || 2 < 1. Consider its generator / e Hol(B,W): 

(i-z!)(i + z 2 ) z 2 {\ - Zl + -4 + ziy 



f( x dF t ( Zl ,z 2 ) 
J(Zi, z 2 ) = - 



dt 



t=0 



1 + zi ' (1 + zi) 2 



It is clear that / has three boundary null points: ri = (1,0), t 2 = (i,0) and 
r 3 = (—i,0). For each one of them we just calculate 

(/(rrQ.n) -(l-r)(l+r 2 ) 
ttl = r-T ,-1 = r-T (l +r) ( r -l) = X > 

a 2 = lim = Um -(l-n)(l-r»)H) = 

r-»i- r — 1 r->i- (l+n)(r — 1) 

(/(rT 3 ),T 3 ) -(l + ri)(l-r 2 )i 
a 3 = lim = lim — — — = -2. 

r-»i- r — 1 r-»i- (1 — ri)(r — 1) 

Hence, the following three inequalities hold simultaneously for all z G B and 
t > 0: 

d Tl ( J F 1 t (z))<e- t d Tl (z), 
<e 2t d T2 (z), 
d T3 (F t (z))<e 2t d T3 (z). 

These inequalities mean that for each point z € B and for each t > 0, the image 
Ft (2) belongs to the intersection of the ellipsoids: 

E( n , e~ l d Tl (z)) f| E(t 2 , e 2t d T2 (z)) f| £(r 3 , e 2 *d T3 (z)). 

Example 4. Consider the one-parameter continuous semigroup S — {F t }t>o, 
consisting of holomorphic self-mappings of the open unit disk A in the complex 
plane, defined by 



F t (z) = 1 - (1 - exp(-i) + exp(-t)Vi - zf, z e A, t > 0. 
One can check that S is generated by the following function: 



f(z) = -2^T~z{VT~z-l). 

It is easy to see that /(l) = 0, but / has no angular derivative at the point 
z = \. At the same time, this point is not even a fixed point of S. 

Example 5. Now we consider the semigroup S = {F t } t > C Hoi (A) defined 
by 

(1 + Z )°(')-(1- Z )«W 
' (l + z)«(*) + (l-z)«(*) ' 
where a(t) = e~ 2t . Differentiating at t = 0, we find its generator: 

1 + z 



f(z) = (I -z z ) log 



1 - z 
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Similarly as in the previous example, /(±1) = 0, but / has no angular derivative 
at the points t\ = 1 and T2 = — 1. However, in contrast with that example, these 
points are fixed points of the semigroup: 

F t (±l) = ±1 for all t > 0. 

Moreover, it is possible to calculate the "non-Euclidean distance" d±i(F t (z)). 
In particular, for real z — x we have: 

Since < a(t) < 1 when t > 0, we conclude that for each fixed t > 0, the 

quotient ( ^ 1 ^ t ^^ tends to infinity as z = x tends to 1 radially, i. e., di(F t (z)) 
di [xj 

does not admit an estimate of the form A(t)d\{z). 
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